Hierarchical Regularization Cascade

We present a hierarchical approach for information sharing among different classification tasks, in multitask and multi-class settings. We propose a top-down iterative method, which begins by posing an optimization problem with an incentive for large scale sharing among all classes. This incentive to share is gradually decreased, until there is no sharing and all tasks are considered separately. The method therefore exploits different levels of sharing within a given group of related tasks, without having to make hard decisions about the grouping of tasks. In order to deal with large scale problems, with many tasks and many classes, we extend our batch approach to online setting and provide regret analysis of the algorithm. Based on the structure of shared information discovered in the joint learning settings, we propose two different knowledge-transfer methods for learning novel tasks. The methods are designed to work within the very challenging large scale settings. We tested our methods extensively on synthetic and real datasets, showing significant improvement over baseline and state-of-the-art methods.